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1. Block ACS Reconstruction Theorem 

Denote A as an unknown dictionary, a £ R Ka x as a 
block vector divided into I\ blocks whose size is a x j3. 
Among K blocks, only s blocks are non-zero, so-called 
active. For example, [1, 2, 0, 0,0, 0,0, 3,0, 0] T can be di¬ 
vided into 5 blocks: [1, 2] T , [0,0] T , [0, 0] T , [0,3] T , [0, 0] T , 
and among them only 2 blocks [1, 2] T , [0,3] T are active, 
where a = 2, /3 = 1, K = 5, s = 2. We call this kind of 
block vector as s-block-sparse vector. 

The block ACS theorem : As for the dictionary A satisfy¬ 
ing block 2.S-R1P [I] with RIP constant 8 < 1, if there exists 
another dictionary B, such that for any s-block-sparse vec¬ 
tor a, we can find a s-block-sparse b satisfying 

Aa=Bb, (1) 

then it follows that 

A = B(P n ® I a )D, (2) 

where / is a permutation matrix and D is a block diagonal 
matrix, and further the size of each block is a x a. 

The block ACS theorem provides the guarantee that 
given enough samples y 1 = Aa 1 , once one finds a dictio¬ 
nary and coefficients modelling measurements in each sam¬ 
ples [y 1 ,..., y N ] well, the ambiguity between recovered 
dictionary and original dicionary is the mix in the blocks 
and mix between blocks. 

Specially, if a = 1, block ACS Theorem degenerates to 
standard ACS therom 0. 

2. Proof of Block ACS Theorem 

Recall in block ACS Theorem, the size of blocks in a 
is a x /?. Let’s first prove the theorem in the case when 
/3 = 1 and once it is proven, the general case /? ^ 1 
is trivial to handle: We can split sparse causes a into 
[ai,..., a^], where a, £ R Ka and then the hypotheses 
Aa = A[ai,...,ap] = B b = _B[bi,..., b^] equals 


to Aai = Bhi, which degenerates to the situation where 

0 = 1 . 

To better understand this theorem and prepare for proof 
in full generality, let us start from a simple case when s = 1. 
Denote ef as a A'-dimensional column vector that has 1 in 
its i-th coordinate and zeros elsewhere. Pick up a block 
vector a' <J = (ef <g> e“), i = l,K. j = 1,..., a, say¬ 
ing that its /-th coordinate in y-th block is 1 and zeros else¬ 
where. For some matrix B and 1-block-sparse b' J , from 
hypothesesQ] it follows that 

Aa« = A(ef ® e“) = Bh" = ® I a ) d y , (3) 

for some mapping 7r : {1,..., K} x {1,..., a} —» {1,..., K} 
and dij £ R“. 

Now, first, let us prove n is only respect to i. From Equa- 
tion[2 we know that for any j ^ k, A( a u + a lk ) = Aa y + 
Aa ik = Bb'i + Bb ik = B{b ij + b ife ). Since a ij + a ik 
is 1-block-sparse, Equation[T|implies that b' 3 + b lk should 
also be 1-block-sparse, which implies that n(i,j) = tr(i, k). 
Hence, it follows that ir : {1,..., AT} —> {1,..., K}, and 

A(ef <8) e“) = B(e% {i) ® / a )d y . (4) 

Then, let us prove that Di = [cU,..., dj a ] is invert¬ 
ible. Denote a 1 = [a* 1 ,..., a la ] and b l = [b* 1 ,..., b la \. 
From Equation IU it follows that Aa x = A [a' 1 ,..., a'"] = 

A[(ef ® e“),..., (ef ® e“)] = A(ef <8 I a ), and Aa 1 = 
Bb l = .B(ef w ®7 0 ) [da,..., die] = ® J„)A- 

Therefore, 

A(ef ®I a ) = ® I a )Di. (5) 

A satisfies block RIP condition, and so rank(A(ef r (8 
la)) = ol. Form Equation 0 rank (B(e^^ <g> I a )Di) = 
a. From basic linear algebra knowledge, we know 
rank(XF) < min(rank(A),rank(F)), for any matrix 
X, Y. So rank(Di) > a. Since Di £ K“ xa , rank(Hi) = 
a. 

Now, let us show n is necessarily injective. Suppose 
7 \{i) = 7r(j), with i ^ j, then from Equation |5j A(ef ® 


1 


(9) 


I a ) = -B(ef (i ) ® I<x)Di = 0 I a )D j D J 1 D i = 

A(e^ 0 Io^DJ 1 Di. Since A satisfies RIP, which implies 
A cannot map 2 different s-block-sparse vector to the same 
measurement, this is only possible if i = j. Thus, 7 r is in¬ 
jective. 

Denote I\ and D as: 


D i • ■ ■ 0 


Pn = 


oK 

MB 


e ir(K) 




(6) 

Since tt is injective, P^ is a permutation matrix. Let us stack 
Equation [5] from left-to-right on both sides, and it follows 
that on left sides, [A(ef 0 I a ), ..., A(e ^ 0 I a )\ = A, and 
on right sides, [5(ef (1) 0l a )D 1 ,..., B{e^ (K) 0l a )d K ] = 
B(Pk 0 I a )D. Hence, finally, we proved in simple case 
A = B(Pt T 0 I a )D. 

Before proving block ACS theorem, we first prove a 
proposition, and then use it for proving block ACS. 

We use the same notation reported in El: Denote 


[K] for the set {1 ,... ,K} and ^ f° r the set of s- 

element subset of [A']. Moreover, denote the dictionary 
A = [Ai,... ,Ak] with A, G R Px “, and SpanjAg} = 


Proposition: Suppose that A satisfies block RIP condi¬ 
tion and that 


k : 




is a mapping with the following property: for all S G 



Span{A 5 } = Span{H« ;(s) }. (7) 

Then, there exist a permutation matrix P K G R K x K and an 
invertible block diagonal matrix D G suc p t jj at 

A = B(P k 0 I a )D. 


Proof. We prove from s to 1 inductively, and the final case 
s = 1 having already been worked out at the simple case. 
First, let’s show function k is injective. Suppose that 

S,S' G are different and k(S) = k(S') holds, 

then by Equation [7] Span{As} = Span{I3 K ( 5 )} = 
Span{£? K (s/)} = SpanjA^/}. From below Lemma 7, it 
turns out that S = S', which implies k is injective. 

Now, let us prove the Proposition inductively. Denote 
l = k -1 as the inverse of k. Fix S = {ii,..., i s _i} G 

^ ^, and set £>i = S U {p} and 1 S 2 = S U {g} for some 

fixed p, q S with p ^ q. (Since s < AT, K — (s — 1) > 1, 
thus, it’s possible to find such p and q.) From Equation [7] 
we obtain: 


( 8 ) 


Span{A t(52) } = Span {B S2 }. 

Let us intersect Equation [ 8 ] and Equation 0 and 
from below Lemma 2 it follows that Spanji?^} Cl 
Span{P 52 } = Span{A t( 5 1 ) nt( 5 2 )}. Since Span{P s } C 
SpanjP^j} fl Span{P 5 2 }, it follows that Span{7?s} C 
Span{A t ( 5 1 ) nt ( 5 2 )}. The number of the elements in t(<Si) Cl 
i(<S> 2 ) is s — 1 , since i(p) ^ i(q), with p q,by injectivity 
of l . Moreover the number of the elements in S is also s — 1, 
which implies that 


Span{7?s} = Span{A t(5l)nt(52) }. 


( 10 ) 


The association S —> l(S±) Cl £(£> 2 ) from Equation [TO] de¬ 


fines a function er : 


, with property 


(S) - (f, 

that Span{P 5 } = Span{A CT ( 5 )}. 

Finally, let’s show that a is injective. Suppose S , S' G 

, and er(iS) = a(S'), it follows that SpanjP^} = 

Span{A CT(S )} = Span{A CT(5 /)} = Span{H S /}, from be¬ 
low Lamma 7, it follows that S = S', which implies a 
is injective. Hence, let £ = o ~ 1 . with properties: for all 


5 G ( j R \ )• Span{A s } = Span{% s) }. 


□ 


Lemma 1: If the dictionay A = [Ai,..., Ak] satisfies 
block 2s-RIP with RIP constant 5 < 1, then for S, S' G 



SpanjA^} = SpanjAg/} => S = S' . (11) 


Proof. Suppose that S 7^ S' G 


[K\ 

s 


satisfying 


Span {.4 5 } = Span {A 5 /}. Then without loss of generality, 
there is an * G S with i qL S', but atoms A, G SpanjA^/}, 
which implies that the RIP constant 5 = 1, a contradiction 
to the assumption on A. □ 


Lemma 2: If the dictionay A satisfies block 2s-RIP with 
RIP constant S < 1, then for S , S' G 

Span{A 5 n <S'} = Span{A 5 } n Span{A 5 }. (12) 

Proof. The inclusion “C” is trivial, so Let us prove “D”. 
Suppose a block vector y G SpanjA^} n Span{Ag 2 }. Ex¬ 
press y as a linear combination of s atoms of A indexed by 
S and, separately, as a combination of s atoms of A indexed 
by S'. By RIP condition, these linear combinations must be 
identical. In particular, y was expressed as a linear com¬ 
bination of atoms of A indexed by S fl S', and thus is in 

Span{ A sn s'} □ 



Span{A t(5l) } = Span{S 5l } 






Proof of block ACS Theorem: Fix<S = {xi, x 2 ,..., x s } £ 



t'j £ Tj , a finite subset of R. Suppose that b is s-block- 
sparse with Aa = B b, and then, b £ Span{e^ ® e “|i = 
1 = 1, for some {pi,p 2 , Viewing 

each s-element subset of [K\ as a color, this map 



is a coloring of the finite set Ti x • • • x Tf with colors in 




C = 


From Ramsey theory 0, it implies that there are 2- 



such that f(t \,..., tf) = {ft, i" 2 , ■■■, ffe} holds for all 
x ... x TL s a . Now, let us define function 

n{{x 1 ,x 2 , :.,x k }) = {ri,r 2 ,..., r}, (15) 

where {ri, r 2 ,..., r^} is generated by above recipe. 

Claim: The mapping k defined by Definition[l5lsatisfies 
Equation |7] 

Proof First let us prove 


Span{4s} C Span{B Aip/lQ(5) }. (16) 


Choose a pair of elements h, h' £ Tl\ x ... x Tl s a , which 
differ only in the /-th coordinate. By construction, the vec¬ 
tors Ah, Ah' are in the right-hand set of Eauation[T 6 l Then, 
the difference A{h — h') that is a nonzero scalar multiple 
of one of bases of the left-hand set of Equation fl 6 l is also 
in the right-hand set of Equation[l 6 ] Hence, every bases of 
the left-hand set of Equation fl 6 l is in the right-hand set of 
Equation[l 6 l i.e.SpanjHs} C Span{_B K (s)}. 

Moreover, the dimension of left-hand set is sK, and 
the dimension of right-hand set is at most sK. Hence, 
Span{A s } = Span{B K ( S )}. □ 

Finally, from the Proposition , block ACS theorem is 
proven. 
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